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GREEDY  APPROXIMATION  WITH  REGARD  TO 
BASES  AND  GENERAL  MINIMAL  SYSTEMS1 

S.V.  Konyagin  and  V.  N.  Temlyakov 


Abstract.  This  paper  is  a  survey  which  also  contains  some  new  results  on  the  nonlinear 
approximation  with  regard  to  a  basis  or,  more  generally,  with  regard  to  a  minimal  system. 
Approximation  takes  place  in  a  Banach  or  in  a  quasi-Banach  space.  The  last  decade  was  very 
successful  in  studying  nonlinear  approximation.  This  was  motivated  by  numerous  applications. 
Nonlinear  approximation  is  important  in  applications  because  of  its  increased  efficiency.  Two 
types  of  nonlinear  approximation  are  employed  frequently  in  applications.  Adaptive  methods 
are  used  in  PDE  solvers.  The  m- term  approximation  considered  here  is  used  in  image  and 
signal  processing  as  well  as  the  design  of  neural  networks.  The  basic  idea  behind  nonlinear 
approximation  is  that  the  elements  used  in  the  approximation  do  not  come  from  a  fixed  linear 
space  but  are  allowed  to  depend  on  the  function  being  approximated.  The  fundamental  ques¬ 
tion  of  nonlinear  approximation  is  how  to  construct  good  methods  (algorithms)  of  nonlinear 
approximation.  In  this  paper  we  discuss  greedy  type  and  thresholding  type  algorithms. 


1.  Greedy  Algorithms  with  regard  to  bases 

Let  a  Banach  space  X  with  a  basis  \F  =  {V’fcjfcLn  be  given.  We  consider  the  following 
theoretical  greedy  algorithm.  For  a  given  element  /  E  X  we  consider  the  expansion 

OO 

(1-1)  /  =  '52ck{f^)^k- 

k= 1 

For  an  element  /  E  X  we  call  a  permutation  p,  p(j)  =  kj ,  j  =  1,2,...,  of  the  positive 
integers  decreasing  and  write  p  E  D(f )  if 

(1-2)  |cfcl(/,W)|>|cfc2(/,W)|>...  . 

In  the  case  of  strict  inequalities  here  D(f )  consists  of  only  one  permutation.  We  define  the 
TO-th  greedy  approximant  of  /  with  regard  to  the  basis  ’F  corresponding  to  a  permutation 
p  E  D(f)  by  formula 

m 

Gm(f)  :=  Gm(f,v)  :=  Gm(/,tf,p)  := 

3  = 1 

1This  research  was  supported  by  the  National  Science  Foundation  Grant  DMS  0200187  and  by  ONR 
Grant  N00014-96-1-1003 
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We  note  that  there  is  another  natural  greedy  type  algorithm  based  on  ordering  || Ck(f, 
instead  of  ordering  absolute  values  of  coefficients.  Denote  A m(/)  a  set  of  indices  such  that 

,  “in  IM/,*)^II  >  .max  \\ck(f,^)^k\\. 

fc£Am(/)  Ai^Am(/) 

We  define  G*(f,  T)  by  formula 

G*(/,  *)  :=  Sa„(/)  (/,*),  where  SE(f)  :=  SB(f,  *)  :=  £  «(/-  *)*■ 

k£E 

It  is  clear  that  in  the  case  of  normalized  basis  (HV’fcll  =  1,  k  =  1,  2, . . . )  the  above  two  greedy 
algorithms  coincide. 

In  the  case  X  =  Lp  we  will  write  p  instead  of  Lp  in  notations.  It  is  a  simple  algorithm 
which  describes  the  theoretical  scheme  (it  is  not  computationally  ready)  for  to- term  ap¬ 
proximation  of  an  element  /.  We  will  call  this  algorithm  Thresholding  Greedy  Algorithm 
(TGA).  In  order  to  understand  the  efficiency  of  this  algorithm  we  compare  its  accuracy 
with  the  best  possible  when  an  approximant  is  a  linear  combination  of  to  terms  from  T.  We 
define  the  best  to- term  approximation  with  regard  to  T  as  follows 

CTrn(/)  :=<rm(f,9)x  :=  inf  \\f  -  Y'ctipkWx, 

C*-A  KA 

where  inf  is  taken  over  coefficients  Ck  and  sets  of  indices  A  with  cardinality  |A|  =  to.  The 
best  we  can  achieve  with  the  algorithm  Gm  is 

\\f-Grn(f,y,p)\\x=arn(f,y)x, 

or  a  little  weaker 

(1-3)  \\f-Gm(f,*,p)\\x<Gvm(f,*)x 

for  all  elements  /  e  X  with  a  constant  G  =  C(X,  T)  independent  of  /  and  to.  It  is  clear 
that  in  the  case  X  =  H  is  a  Hilbert  space  and  T  is  an  orthonormal  basis  we  have 

Let  us  begin  our  discussion  by  an  important  class  of  bases:  wavelet  type  bases.  De¬ 
note  %  :=  {-HfcjfcT i  the  Haar  basis  on  [0,1)  normalized  in  L2(0, 1).  We  denote  by  7~LP  := 
{Hk^p}(k=i  the  Haar  basis  %  renormalized  in  Lp( 0, 1).  We  will  use  the  following  definition 
of  the  Lp-equivalence  of  bases.  We  say  that  T  =  {^k}kLi  is  Tp-equivalent  to  $  =  {4>k}kLi 
if  for  any  finite  set  A  and  any  coefficients  q,  fc  6  A,  we  have 

Gi(p,T,$)||  <  ||  J^CfcV’fcllp  <  G2(p,W,$)||  ^cfc0fc||p 

k£  A  k£  A  fee  A 

with  two  positive  constants  Ci(p,  <f>),  C2(p,  \l/,  <f>)  which  may  depend  on  p ,  T,  and  <f>. 
For  sufficient  conditions  on  T  to  be  Lp-equivalent  to  %  see  [FJ]  and  [DKT],  In  particular, 
it  is  known  that  all  reasonable  univariate  wavelet  type  bases  are  Lp-cqui  valent  to  'H  for 
1  <  p  <  oo.  We  proved  the  following  theorem  in  [T2]. 
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Theorem  1.1.  Let  1  <  p  <  oo  and  a  basis  4>  be  Lp-equivalent  to  the  Haar  basis  TL.  Then 
for  any  f  G  Lp( 0,1)  we  have 

\\f-G^(f,n\P<C(p^)am(f^)p 

with  a  constant  C(p ,  41)  independent  of  f  and  m. 

By  a  simple  renormalization  argument  one  obtains  the  following  version  of  Theorem  1.1. 

Theorem  1.1A.  Let  1  <  p  <  oo  and  a  basis  41  be  Lp-equivalent  to  the  Haar  basis  TLp. 
Then  for  any  f  G  Lp( 0, 1)  and  any  p  G  D(f)  we  have 

||/  -  Gm(f,  W,  p)\\p  <  C(p ,  *  )P 

with  a  constant  C(p ,  41)  independent  of  f ,  p,  and  to. 

We  note  that  [T2]  also  contains  a  generalization  of  Theorem  1.1  to  the  multivariate  Haar 
basis  obtained  by  the  multiresolution  analysis  procedure.  These  theorems  motivated  us  to 
consider  the  general  setting  of  greedy  approximation  in  Banach  spaces.  We  concentrated  on 
studying  bases  which  satisfy  (1.3)  for  all  individual  functions. 

Definition  1.1.  We  call  a  basis  41  greedy  basis  if  for  every  f  G  X  there  exists  a  permutation 
p  G  D(f)  such  that 

(1-4)  \\f-Gm(f,*,p)\\x<Gvm(f,*)x 

holds  with  a  constant  independent  of  f,  m. 

The  following  proposition  has  been  proved  in  [KT1]. 

Proposition  1.1.  If  41  is  a  greedy  basis  then  (1-4)  holds  for  any  permutation  p  G  D(f). 

Theorem  1.1A  shows  that  each  basis  41  which  is  Lp-equivalent  to  the  univariate  Haar 
basis  TLP  is  a  greedy  basis  for  Lp( 0, 1),  1  <  p  <  oo.  We  note  that  in  the  case  of  Hilbert  space 
each  orthonormal  basis  is  a  greedy  basis  with  a  constant  G  =  1  (see  (1.4)). 

We  give  now  the  definitions  of  unconditional  and  democratic  bases. 

Definition  1.2.  A  basis  41  =  {'f’kjkLi  of  a  Banach  space  X  is  said  to  be  unconditional  if 
for  every  choice  of  signs  0  =  {0k}kLi,  0k  =  1  or  —  1,  k  =  1,2, ,  the  linear  operator  Mg 
defined  by  Mg(Y^k=  l  afcV’fe)  =  J2kLi  Ok0kfi>k  is  a  bounded  operator  from  X  into  X. 

Definition  1.3.  We  say  that  a  basis  41  =  {fi>k}kLi  is  a  democratic  basis  for  X  if  there 
exists  a  constant  D  :=  D(X,  41)  such  that  for  any  two  finite  sets  of  indices  P  and  Q  with 
the  same  cardinality  \P\  =  \Q\  we  have  ||  X^epV’fcll  <  D\\  YlkeQ  ^fcH- 

We  proved  in  [KT1]  the  following  theorem. 
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Theorem  1.2.  A  basis  is  greedy  if  and  only  if  it  is  unconditional  and  democratic. 

This  theorem  gives  a  characterization  of  greedy  bases.  Further  investigations  ([T3], 
[CDH],  [KP],  [GN])  showed  that  the  concept  of  greedy  bases  is  very  useful  in  direct  and 
inverse  theorems  of  nonlinear  approximation  and  also  in  applications  in  statistics.  The 
papers  [KT1],  [T2],  contain  other  results  on  greedy  bases. 

Let  us  discuss  a  question  of  weakening  the  property  of  a  basis  of  being  a  greedy  basis. 
We  begin  with  a  concept  of  quasi-greedy  basis. 

Definition  1.4.  We  call  a  basis  HI  quasi-greedy  basis  if  for  every  f  G  X  and  every  permu¬ 
tation  p  G  D(f)  we  have 

(1-5)  \\Gm(f,*,p)\\x  <C\\f\\x 

with  a  constant  C  independent  of  f ,  m,  and  p. 

It  is  clear  that  (1.5)  is  weaker  then  (1.4).  P.  Wojtaszczyk  [W]  proved  the  following 
theorem. 

Theorem  1.3.  A  basis  H>  is  quasi-greedy  if  and  only  if  for  any  f  G  X  and  any  p  G  D(f) 
we  have 


(1.6) 


11/  -  Gm(/,  ^,p)l|  0  as  m  >  co. 


We  proceed  to  an  intermediate  concept  of  almost  greedy  basis.  This  concept  has  been 
introduced  and  studied  in  [DKKT].  Let 

OO 

/  =  J^cfc(/)V>fc. 

k= 1 


We  define  the  following  expansional  best  to- term  approximation  of  / 

°m(f)  ■■=  :=  inf  ||/- y'cfc(/)V’fe||. 

A’|A|=m  fegA 


It  is  clear  that 

<7m(/,  ’£)  <  cfm(/,  '£)• 
It  is  also  clear  that  for  an  unconditional  basis  HI  we  have 


Definition  1.5.  We  call  a  basis  H>  almost  greedy  basis  if  for  every  f  G  X  there  exists  a 
permutation  p  G  D(f)  such  that 

(1-7)  \\f-Gm(f,*,p)\\x<Cirm(f,*)x 

holds  with  a  constant  independent  of  f ,  to. 

The  following  proposition  follows  from  the  proof  of  Theorem  3.3  of  [DKKT]  (see  Theorem 
1.4  below). 
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Proposition  1.2.  If  <3/  is  an  almost  greedy  basis  then  (1.7)  holds  for  any  permutation 

pG  £>(/). 

The  following  characterization  of  almost  greedy  bases  has  been  obtained  in  [DKKT]. 

Theorem  1.4.  Suppose  T  is  a  basis  of  a  Banach  space.  The  following  are  equivalent: 

A.  T  is  almost  greedy. 

B.  T  is  quasi-greedy  and  democratic. 

C.  For  any  (respectively,  every)  A  >  1  there  is  a  constant  C  =  C\  such  that 

\\f-G[Xm](f,*)\\<Cxvm(f,*). 

We  will  prove  an  estimate  for  dn(f,  T)  in  terms  of  am(f,  T)  for  a  quasi-greedy  basis  T. 
For  a  basis  T  we  define  the  fundamental  function 


<p(m)  :=  sup  ||£>||. 
l^l<™  keA 


We  also  need  the  following  function 


4>{m) 


inf 

|  A\=m 


k£A 


It  will  be  convenient  to  define  the  quasi-greedy  constant  K  to  be  the  least  constant  such 
that 

\\Gm(f)\\  <  K\\f\\  and  \\f  -  Gm(f)\\  <  K\\f\\,  f  e  X. 

We  will  prove  an  inequality  that  has  been  obtained  in  [DKKT]. 

Theorem  1.5.  Let  T  be  a  quasi-greedy  basis.  Then  for  any  m  and  r  there  exists  a  set  E, 
\E\  <  m  +  r  such  that 


11/  -  sE(f)\\  <  c(i  + 

Proof.  If  <7m(/)  =  0  then  /  =  \A\  <  m  and,  therefore,  SA(f)  =  f.  Let 

am(/)  /  0  and  A  be  a  set,  \A\  =  m,  such  that 

(1-8)  11/ -Pm(/)||  <  2dm(/),  pm(f)  =  bkifk- 

k£A 

Denote  g  :=  f  —  Pm{f)-  Let  B,  \B\  =  r,  be  such  that 

Gr(g)  =  ck{g)i>k. 

k£B 
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Consider 


(1-9)  /  -  SaubU)  =  9~  Saub(9)  =9~  SB{g )  -  SA\B{g). 

By  the  assumption  that  T  is  quasi-greedy  and  by  the  definition  of  B  we  get 
(1-10)  \\9-SB(g)\\<C1\\g\\<2C1am(f). 

Let  us  estimate  ||<S'J4\s(g)||.  By  Lemma  2.2  from  [DKKT]  we  get 

max  \ck(g)\  <  4Jl 2(0(r  +  l))^1 1|^||. 

k£A\B 

Next,  by  Lemma  2.1  from  [DKKT]  we  obtain 

(1-11)  1 1  ^A\B  (fl1)  1 1  <  (2  K)3(p(m)0(r  +  l)_1y|. 

Combining  (1.10)  and  (1.11)  we  derive  from  (1.9)  for  E  :=  A  U  B 

\if-sEm<c(i  +  -0L)rTm{f), 

Theorem  1.5  is  proved. 


2.  Weak  Greedy  Algorithms  with  regard  to  bases 

The  following  weak  type  greedy  algorithm  was  considered  in  [T2].  Let  t  E  (0,1]  be  a 
fixed  parameter.  For  a  given  basis  T  and  a  given  /  E  X  denote  A m(t)  any  set  of  to  indices 
such  that 

(2.1)  min  |cfc(/,T)|>t  max  |cfc(/,T)| 

fc£Am(i) 

and  define 

GlU)  ~  GUf,*)  ~  E  <*(/>*)*• 

fee  Am  (t) 

We  call  it  the  Weak  Thresholding  Greedy  Algorithm  (WTGA)  with  the  weakness  sequence 
{t}.  It  was  proved  in  [T2]  that  in  the  case  of  X  =  Lp,  1  <  p  <  oo,  and  T  is  the  Haar  system 
1~LP  normalized  in  Lp  we  have  for  any  /  G  Lp 

(2.2)  ||/  -  Gl(f,Hp) |U„  <  C(p,t)am(f,Hp) t„. 

We  note  here  that  the  proof  of  (2.1)  from  [T2]  works  for  any  greedy  basis  instead  of  the 
Haar  system  7~Lp.  Thus  for  any  greedy  basis  $  of  a  Banach  space  X  and  any  t  E  (0, 1]  we 
have  for  each  /  E  X 

(2-3)  ||/  -  GUf,  n\x  <  C(9,t)<rm(f,  *)x. 

This  means  that  for  greedy  bases  we  have  more  flexibility  in  constructing  near  best  to- term 
approximants. 

We  now  consider  the  Weak  Thresholding  Greedy  Algorithm  with  regard  to  a  quasi-greedy 
basis  T.  The  following  theorem  is  essentially  due  to  Wojtaszczyk  [W]. 
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Theorem  2.1.  Let  ^  be  a  quasi- greedy  basis  for  a  Banach  space  X .  Then  for  any  fixed 
t  e  (0, 1]  we  have  for  each  f  E  X  that 

GL(/^)->/  as  to  — >  oo. 


Proof.  Let 


Denote 


and 


GL(/,*)=  E  Cj(/)*  =  SA„(*)(/.*)- 

j€Am(t) 


a  :=  x  lci(/)l 

J  ^-^-m  (£) 


Am  :=  {j  :  |Cj(/)|  >  Ct}  £  Am(i), 
A  2m:=  {j:|Cj(/)|>to}DAm(t). 


Thus  we  have 


Sa„W(/.  * )  =  SA,Jf.  ¥)  +  SAm(tnA,Jf,  ¥). 
The  assumption  that  T  is  quasi-greedy  implies  that 


We  will  prove  that 


S^if,  ’L)  ->■  /  as  m->oo. 
I^Am(t)\A^(/,^)ll  ->  0  as  TO^OO. 


We  note  that 


(2.4) 


SA„<t,\AJ,(/,*)  =  SA„(t)\Al,( 


E 

j:ta<\cj(f)\<a 


Cj{f)*j,*). 


We  need  a  lemma  on  properties  of  quasi-greedy  systems. 

Lemma  2.1.  Let  \&  be  a  quasi-greedy  basis.  Then  for  any  two  finite  sets  of  indices  A  C  B 
and  coefficients  0  <  t  <  \aj  <  1 ,  j  E  B,  we  have 

W^ajfjW  <  C(X,y,t)\\J2aj^j\\- 

j£A  j£B 


Proof.  The  proof  is  based  on  the  following  known  lemma  (see  [DKKT])  that  is  essentially 
due  to  Wojtaszczyk  [W]. 
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Lemma  2.2.  Suppose  T  is  a  quasi-greedy  basis  with  a  quasi-greedy  constant  K .  Then  for 
any  real  numbers  aj  and  any  finite  set  of  indecies  P  we  have 


(4  K2)  1  minimi  ||  <  ||  a^j  ||  <  2K  max \aj  1 1|  E  H- 

3  j£P  j£P  J  j£P 

Using  this  lemma,  we  get 

\\J2aM\  <  2K\\  E  ^-11  ^  (2^)2n  E^-ii  ^  E  a^-ii- 

j£A  jeA  j£B  j£B 

This  proves  Lemma  2.1. 

We  continue  the  proof  of  Theorem  2.1.  Denote 

■=  E  cAf)^i- 

j:ta<\cj(f)\<a 

Then  by  Lemma  2.1  we  get  from  (2.4) 

ll^(t)\Aii(/,W)||<C'||/a||. 

It  remains  to  remark  that  a->0asm-HX)  and  fa  — >  0  as  a  — *  0. 

We  note  that  the  mth  greedy  approximant  Gm(f ,  T)  changes  if  we  renormalize  the  system 
{' xf>n }  (replace  it  by  a  system  {A n'f’n})-  This  gives  us  more  flexibility  in  adjusting  a  given 
system  {i/jn}  for  greedy  approximation. 

Let  us  now  proceed  to  an  almost  greedy  basis  T.  Similarly  to  the  proof  of  Theorem  2.1 
one  can  prove  the  following  lemma. 

Lemma  2.3.  Let  T  be  a  quasi-greedy  basis.  Then  for  a  fixed  t  E  (0, 1]  and  any  m  we  have 
for  any  f  E  X 

||G(n(/,T)||<C(f)||/||. 

Theorem  2.2.  Let  T  be  an  almost  greedy  basis.  Then  for  t  E  (0, 1]  we  have  for  any  m 
(2-5)  \\f-Gtm{f)\\<C{t)<rm(f). 

Proof.  Take  any  e  >  0  and  find  P,  |P|  =  m  such  that 

Il/-M/)II  <*m(/)  +  e. 

Let  Q  :=  A m(f)  with  A m(t)  from  the  definition  of  G(n(/).  Then 

(2.6)  ||/  -  GUm  <11  /  -  Sp(f) II  +  II Sp(f)  -  SQ(f) ||. 
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We  have 


(2-7)  SP(f)  -  SQ(f)  =  SP\Q(f )  -  SQ\P(f). 

Let  us  estimate  first  ||5,g\p(/)||.  Denote  fi  ’■=  f  —  Sp(f).  Then 

SQ\P{f)  =  SQ\P(f  i). 


Next 


min 

k£Q\P 


Ck{fl) 


min 

k£Q\P 


|cfc(/)|  >  min|cfc(/)|  > 
keQ 


fmax  |cfc(/)|  >  fmax  |cfc(/i)|  =  t  max  |cfc(/i)|. 

HQ  HQ  HQ\P 

Thus  Q  \  P  =  An(t)  for  /i  with  n  :=  \Q  \  P |.  By  Lemma  2.3  we  have 

(2-8)  ||Sg\p(/)||<C'1(f)||/1||. 


We  now  estimate  ||5,p\g(/)||.  From  the  definition  of  Q  we  easily  derive 


(2-9) 


at  <  b  where  a  :=  max  I cAf)\,  b  :=  min  \ck{f)\- 

k£P\Q  k£Q\P 


By  Lemma  2.2  (see  Lemma  2.1  from  [DKKT]) 

(2.10)  l|SpXQ(/)||  <  2Ka\\  Y.  *11 

k€P\Q 


and  (see  Lemma  2.2  from  [DKKT]) 

(2.11)  I|Sq\p(/)II  >  (4A’2)-16||  Y  *11- 

k£Q\P 

By  Theorem  1.4  an  almost  greedy  basis  is  a  democratic  basis.  Thus  we  get 

(2-12)  ||  ]T  M<D\\  E  ^H- 

k£P\Q  k£Q\P 

Combining  (2.6)  (2.12)  we  obtain  (2.5).  Theorem  2.2  is  proved. 

We  now  discuss  a  stability  of  greedy  type  property  of  a  basis.  Let  0  <  a  <  A&  <  b  <  oo, 
k  =  1,2,...  and  for  a  basis  T  =  {V’fc}  consider  <FA  :=  {A fcV’fc}- 
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Theorem  2.3.  Let  a  basis  T  have  one  of  the  properties 

1.  Greedy. 

2.  Almost  greedy. 

3.  Quasi-greedy. 

Then  the  basis  \I/A  has  the  same  property. 

Proof.  Let  /  el  and 

/  =  ^  ck(f)\k1\kif>k. 

k  k 

Consider  _ 

Gm(f^x)=  (cfc(/)Afc  1)Afc,0fc- 

&£  Am 

Then  we  have 

min  \ck(f)\  >  a  min  \ck(f)  (A^1  >  a  max  |cfc(/)  |  A^T1  >  j  max  \ck{f)\. 

Therefore,  the  set  Am  can  be  interpreted  as  a  A m(t)  with  t  =  a/b  with  regard  to  the  basis 
T.  It  remains  to  apply  the  corresponding  results  for  G^(/,  T):  (2.3)  in  the  case  1,  Theorem 
2.2  in  the  case  2,  and  Theorem  2.1  in  the  case  3.  This  completes  the  proof  of  Theorem  2.3. 

In  the  paper  [KaT]  the  following  modification  of  the  above  weak  type  greedy  algorithm  in 
a  way  of  further  weakening  the  restriction  (2.1)  has  been  studied.  We  call  this  modification 
the  Weak  Thresholding  Greedy  Algorithm  (WTGA)  with  a  weakness  sequence  r  =  {tk}. 
Let  a  weakness  sequence  r  :=  {tfcjfcTi,  tk  £  [0, 1],  k  =  1, . . .  be  given.  We  define  the  WTGA 
by  induction.  We  take  an  element  /  £  X  and  at  the  first  step  we  let 

Ai(t)  :=  {m};  G\(f,  T)  :=  cni  ipn, 

with  rii  any  satisfying 

l^ni  |  ^  max  | Cfi  | 

n 

where  we  denote  for  brevity  cn  :=  cn(f,  T).  Assume  we  have  already  defined 

GTm-M  <p)  :=  4?)  :=  £ 

n£Am_i(r) 

Then  at  the  mth  step  we  define 

A m(r)  :=  Am_i(r)  U  {nm};  GTm{f ,  T)  :=  G%T(f,  T)  :=  cn^n 

n£Am(r) 

with  nm  f  Am_i(r)  any  satisfying 

|cnm|  —  max  |cn|. 
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Thus  for  an  /  G  X  the  WTGA  builds  a  rearrangement  of  a  subsequence  of  the  expansion 
(1.1).  If  T  is  an  unconditional  basis  then  always  GTm(f ,  T)  —>■/*.  It  is  clear  that  in  this  case 
/*  =  /  if  and  only  if  the  sequence  {rikf^Li  contains  indices  of  all  nonzero  cn(/,  T).  We  say 
that  the  WTGA  corresponding  to  T  and  r  is  convergent  (converges)  if  for  any  realization 
GTm  (f,  T)  we  have 

11/ -  -»■  0  as  m^oo 

for  all  feX. 

In  [KaT]  the  following  three  theorems  on  convergence  of  the  WTGA  have  been  proved. 
The  first  one  deals  with  an  arbitrary  Banach  space  X  and  any  basis  T. 

Theorem  2.4.  Let  X  be  a  Banach  space  with  a  normalized  basis  T.  Let  r  =  {tn,n  >  1}  be 
a  weakness  sequence.  The  following  condition  (D)  is  a  necessary  condition  for  the  WTGA 
corresponding  to  T  and  r  to  be  convergent. 

(D)  For  each  subsequence  {rik,  k  >  1}  of  different  indices,  the  series  Y2k=i  tkf>nk  diverges 
in  X . 

If  the  basis  T  is  unconditional,  then  the  above  condition  (D)  is  also  a  sufficient  condition 
for  the  WTGA  corresponding  to  T  and  r  to  be  convergent. 

In  the  case  X  =  Lp([0,  l]d)  one  can  derive  from  Theorem  2.4  a  more  specific  condition  in 
terms  of  r  (see  [KaT]). 

Theorem  2.5.  Let  2  <  p  <  oo,  d  >  1  and  let  T  be  a  normalized  unconditional  basis  in 
Lp{[ 0,  l]d).  Let  r  =  {tn,n  >  1}  be  a  weakness  sequence.  Then  the  WTGA  corresponding  to 
T  and  r  converges  if  and  only  if  r  lP- 

There  is  no  simple  criterion  in  terms  of  r  in  the  case  X  =  Lp([ 0,  l]d),  1  <  p  <  2  and 
arbitrary  unconditional  basis  T.  In  this  case  [KaT]  contains  the  following  result  for  the 
multivariate  Haar  basis  Tip  defined  as  the  tensor  product  of  the  univariare  Haar  bases: 
Ftp  :=  Ftp  x  •  •  •  x  Ftp.  To  formulate  the  result,  introduce  the  following  notation.  For  a 
sequence  {tk,k  >  1}  of  nonnegative  numbers  such  that  lim^oo  tk  =  0,  {t*k,k  >  1}  is  a 
nonincreasing  rearrangement  of  the  subsequence  { tnk ,  k  >  1}  consisting  of  positive  elements 
of  {tk,  k  >  1}. 

Theorem  2.6.  Let  d  >  1  and  1  <  p  <  2.  The  WTGA  corresponding  to  Ftp  and  a  weakness 
sequence  r  converges  in  Lp([ 0,  l]d)  if  and  only  if  one  of  the  following  conditions  is  satisfied: 

(i)  The  sequence  r  =  {tk}  does  not  converge  to  0. 

(ii)  Hindoo  tk  =  0  and 


(2.13)  W°t,kfl-W/r  ‘  =  oo. 

k=  1 

Along  with  convergence  of  the  WTGA  efficiency  of  approximation  by  GTm{-,  T)  has  been 
studied  in  [KaT].  The  accuracy  of  the  WTGA  was  compared  with  best  to- term  approxi¬ 
mation.  In  the  case  of  greedy  basis  and  r  =  {t},  t  e  (0, 1]  the  relation  (2.3)  shows  that 
GTm{-,  T)  realizes  near  best  to- term  approximation.  There  are  two  natural  ways  of  adapting 

ll 


(2.3)  to  the  case  of  nongreedy  basis  or  to  the  case  of  general  weakness  sequence.  In  the  first 
way  (see  [T5],  [T3],  [W],  [Os])  we  write  (2.3)  in  the  form 


and  look  for  the  best  (in  the  sense  of  order)  constant  C(m,r ,  T). 

We  now  formulate  the  correspoding  results  from  [KaT].  For  a  basis  T  we  define  the 
fundamental  function  (p(m)  and  the  function  /(to)  like  in  Section  1.  We  also  need  the 
following  function 

<ps{m)  :=  sup  \\^2^k\\- 
\A\=m  keA 


It  is  clear  that 


(p(m)  =  sup  (ps(n ). 

n<m 


We  now  introduce  some  characteristics  of  a  basis  with  respect  to  a  weakness  sequence  r. 
For  a  subset  V  C  [l,m]  of  integers  we  define 


V)  :=  inf 

{fci} 


i£V 


where  inf  is  taken  over  all  sets  {/q}  of  different  indices.  For  two  integers  1  <  n  <  m  we 
define 

d>(r,m,n)  :=  inf  6(r,m,V ), 

\V\=n,VC[l,m] 


and  finally 


p(r,  m )  :=  sup 

n<m 


<Ps(n) 

0(r,  m,  n) 


The  following  result  has  been  proved  in  [KaT]. 


Theorem  2.7.  Let  T  be  a  normalized  unconditional  basis  for  X .  Then  we  have 

||/  -  CTm(f,  * )||  <  m)am(f ,  T). 

In  Theorem  2.7  we  compare  efficiency  of  G^(-,  T)  with  am(-,  T).  It  is  known  in  approx¬ 
imation  theory  that  sometimes  it  is  convenient  to  compare  efficiency  of  an  approximating 
operator  which  is  characterized  by  m  parameters  with  best  possible  approximation  corre¬ 
sponding  to  smaller  number  of  parameters  n  <  m.  We  use  this  idea  in  approximation  by 
the  WTGA.  Let  us  discuss  a  setting  (see  [KaT])  when  we  write  (2.3)  in  the  form 

and  look  for  the  best  (in  the  sense  of  order)  sequence  {vm}  that  is  determined  by  the 
weakness  sequence  r  and  the  basis  T.  We  need  some  more  notation.  Define 

N 

0(tj  N)  :=  0(t,  N ,  [1,  N])  =  inf  || 

j=i 

Assume  that  0(r,  N )  — >  oo  as  N  — >  oo  and  denote  vm  the  smallest  N  satisfying 

/(r,  N )  >  2 

There  is  the  following  result  ([KaT])  in  this  case. 
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Theorem  2.8.  For  any  normalized  unconditional  basis  we  have 


It  is  interesting  to  compare  this  result  with  some  recent  results  from  [DKKT].  It  has  been 
established  in  [DKKT]  (see  Theorem  1.4  of  present  paper)  that  the  inequalities 

(2.14)  ||/  -  G[Xm](f,  tf)||  <  C/*,  A 4/) 

with  fixed  A  >  1  are  characteristic  for  a  class  of  almost  greedy  bases.  It  is  clear  that  each 
greedy  basis  is  an  almost  greedy  basis.  There  is  an  example  (see  [KT1,  sections  3.3,  3.4]) 
of  almost  greedy  basis  that  is  not  a  greedy  basis.  This  means  that  A  >  1  needed  for  (2.14) 
can  not  be  replaced  by  A  >  1. 


3.  Thresholding  type  approximation  with  regard  to  minimal  systems 

Let  X  be  a  quasi-Banach  space  (real  or  complex)  with  the  quasi-norm  ||  •  ||  such  that  for 
all  x,  y  G  X  we  have  \\x  +  y\\  <  o;( 1 1 m 1 1  +  ||j/||)  and  ||fx||  =  |t|||x||.  It  is  well-known  (see  [KBR, 
Lemma  1.1])  that  there  is  a  p,  0  <  p  <  1,  such  that 


(3.1) 


1/p 


Let  {en}  Clbea  complete  minimal  system  in  X  with  the  conjugate  (dual)  system  {e*  }  C 
X*.  We  assume  that  supn  ||e*  ||  <  oo.  This  implies  that  for  each  x  G  X  we  have 


(3.2) 


lim  e*(x)  =  0. 


Any  element  x  G  X  has  a  formal  expansion 


(3.3) 


x  ~  yX;(z)e„, 

n 


and  various  types  of  convergence  of  the  series  (3.3)  can  be  studied.  In  this  section  we 
deal  with  greedy  type  approximations  with  regard  to  the  system  {en}.  We  note  that  in 
this  section  we  use  the  notations  x  and  {en}  for  an  element  and  for  a  system  respectively 
different  from  the  notations  /  and  T  in  the  previous  sections  to  emphasize  that  we  are  in  a 
more  general  setting  now.  It  will  be  convenient  for  us  to  define  a  unique  ”  greedy  ordering” 
in  this  section.  For  any  x  G  X  we  define  the  greedy  ordering  for  x  as  the  map  p  :  N  — *  N 
such  that  {j  :  e*  ( x )  ^  0}  C  p(N)  and  so  that  if  j  <  k  then  either  Ku)W  >  K(k)(x)\ 01 
lep(j')(IC)l  =  lep(fc)(:r:)l  and  P(j)  <  p(k).  The  rn-th  greedy  approximation  is  given  by 


Gm(jx)  ■  (jrm (a;,  {en})  .  ^ 

j= 1 
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The  system  {en}  is  called  a  quasi-greedy  system  (see  [KT1])  if  there  exists  a  constant  C 
such  that  ||Gm(a:)||  <  C'||x||  for  all  x  e  X  and  me  N.  Wojtaszchyk  [W]  proved  that  these  are 
precisely  the  systems  for  which  lim™-^  Gm(x)  =  x  for  all  x.  If  a  quasi-greedy  system  {en} 
is  a  basis  then  we  say  that  {en}  is  a  quasi-greedy  basis.  It  is  clear  that  any  unconditional 
basis  is  a  quasi-greedy  basis.  We  note  that  there  are  conditional  quasi-greedy  bases  {en} 
in  some  Banach  spaces  [KT,  W].  Hence,  for  such  a  basis  { en }  there  exists  a  permutation  of 
{en}  which  forms  a  quasi-greedy  system  but  not  a  basis.  This  remark  justifies  the  study  of 
the  class  of  quasi-greedy  systems  rather  than  the  class  of  quasi-greedy  bases. 

Greedy  approximations  are  close  to  thresholding  approximations  (sometimes  they  are 
called  “thresholding  greedy  approximations”).  Thresholding  approximations  are  defined  as 

Te(x)  =  ^  e*(x)ej,  e  >  0. 

\e*(x)\>e 

Clearly,  for  any  e  >  0  there  exists  an  m  such  that  Te(x)  =  Gm(x).  Therefore,  if  {en}  is  a 
quasi-greedy  system  then 

(3.4)  \/x  £  X  lim  Tf  (x)  =  x. 

£—>■0 

Conversely,  following  Remark  from  [W,  pages  296  297],  it  is  easy  to  show  that  the  condition 
(3.4)  implies  that  {en}  is  a  quasi-greedy  system. 

Similarly  to  the  above,  one  can  define  the  Weak  Thresholding  Approximation.  Fix  t  G 
(0, 1).  For  s  0  denote 

Dt,e{x)  :=  {j  :  te  <  \e*(x)\  <  e}. 

The  Weak  Thresholding  Approximations  are  defined  as  all  possible  sums 

Tc,d(x)  =  ej(x)eJ  +  ej(x)ei' 

|e*(a:)|>e  j£D 

where  D  C  Dt,e{x).  We  say  that  the  Weak  Thresholding  Algorithm  converges  for  x  G  X 
and  write  x  G  WT{en}(t)  if  for  any  D(e)  C  Dt 

li m  T  D{e)(x)  =  x. 

It  is  clear  that  the  above  relation  is  equivalent  to 

lim  sup  1 1  a:  —  Tetjj(x)\\  =  0. 

e^°  DCDt,e{x) 

We  proved  in  [KT2]  (see  Theorem  3.1  below)  that  the  set  WT{en}(t)  does  not  depend  on 
t.  Therefore,  we  can  drop  t  from  the  notation:  WT{en}  =  WT{en}(t). 

It  turns  out  that  the  Weak  Thresholding  Algorithm  has  more  regularity  than  the  Thresh¬ 
olding  Algorithm:  we  will  see  that  the  set  WT{en}  is  linear.  On  the  other  hand,  by  “weak¬ 
ening”  the  Thresholding  Algorithm  (making  convergence  stronger)  we  do  not  narrow  the 
convergence  set  too  much.  It  is  known  that  for  many  natural  classes  of  sets  Y  C  X  the 
convergence  of  Te(x)  to  x  for  all  x  G  Y  is  equivalent  to  the  condition  Y  C  WT{en}.  In 
particular,  it  can  be  derived  from  [W,  Proposition  3]  that  the  two  above  conditions  are 
equivalent  for  Y  =  X. 

We  suppose  that  X  and  {en}  satisfy  the  conditions  stated  in  the  beginning  of  this  section. 
The  following  two  theorems  have  been  proved  in  [KT2]. 
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Theorem  3.1.  Let  t,t'  G  (0, 1),  x  G  X .  Then  the  following  conditions  are  equivalent: 

1)  limg^o  suPocDt,E(iE)  II TeMx)  ~x\\  =°>' 

2)  limg^o  Te  (x)  =  x  and 


(3.5) 


lim  sup 

s^°  DCDt^(x) 


J2e*j(X)ej 

j£D 


0; 


3)  limg^o Te(x)  =  x  and 


(3.6) 


lim  sup  ||  aje*(x)ej  ||  =  0; 

la#i<1(je-Dt,e(aO)  jsDt,c(x) 


4)  limg^o  Te(x)  =  x  and 

(3.7)  lim  sup  ||  ej  11=0; 

^°\bj\<eU:\e*(x)\>e)  j;  (:j(;r)  >s 

5)  limg^o  sup Dc.Dti  s(x)  II Te,D(x)  -  x\\  =  0. 

So,  the  set  WT{en}(t)  defined  above  is  indeed  independent  of  t  G  (0, 1). 

Theorem  3.2.  The  set  WT{en}  is  linear. 

Let  us  discuss  relations  between  the  Weak  Thresholding  Algorithm  Te ^  (x)  and  the  Weak 
Thresholding  Greedy  Algorithm  Gtm(x).  We  define  Gtrn{x)  with  regard  to  a  minimal  system 
{en}  in  the  same  way  as  it  was  defined  for  a  basis  ’L.  For  a  given  system  {en}  and  t  G  (0, 1] 
we  denote  for  x  G  X  and  to  G  N  by  Wm(t)  any  set  of  to  indices  such  that 

(3.8)  min  \e*(x)\>t  max  |e*(x)| 

iewm(t)  3  3 

and  define 

Glnix)  :=  (^{x,  {en})  :=  SWm{t)(x)  :=  ej(x)e3- 

j€Wm(t) 

It  is  clear  that  for  any  t  G  (0, 1]  and  any  D  C  Dt>e(x)  there  exist  to  and  Wm{t)  satisfying 

(3.8)  such  that 

T6jd{x)  =  <SWm(i)  ix)- 

Thus  the  convergence  Gtm(x)  — *  x  as  to  — >  oo  implies  the  convergence  T€jd(x)  — *  x  as 
e  — *  oo  for  any  t  G  (0, 1].  We  will  now  prove  (see  [KT2],  Proposition  2.2)  that  for  t  G  (0, 1) 
the  inverse  is  also  true. 

Proposition  3.1.  Let  t  G  (0, 1)  and  x  G  X.  Then  the  following  two  conditions  are  equiva¬ 
lent: 

(3.9)  lim  sup  ||Tej£>(a;)  —  ic||  =  0; 

£^°dca,£W 
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(3.10) 


Grn(X)  -  Z||  =  0 


lim 

ra— o 


for  any  realization  G^fx). 

Proof.  The  implication  (3.10)  (3.9)  is  simple  and  follows  from  a  remark  preceding  Propo¬ 
sition  3.1.  We  prove  that  (3.9)  (3.10).  Denote 

em  :=  max  \e*{x)\. 


Clearly  em  — >  0  as  m  — >  oo.  We  have 

(3.11)  G^x)  =  T2€m(x)  +  e*j(x)ej 

j£Dm 

with  Dm  having  the  following  property:  for  any  j  G  Drn 

tem  <  \e*j{x)  \  <  2em. 

Thus  by  condition  5)  from  Theorem  3.1  for  t'  =  tj 2  we  obtain  (3.11). 

Proposition  3.1  is  now  proved. 

Proposition  3.1  and  Theorem  3.1  imply  that  the  convergence  set  of  the  Weak  Thresholding 
Greedy  Algorithm  G^(-)  does  not  depend  on  t  G  (0,1)  and  coincides  with  WT{en}.  By 
Theorem  3.2  this  set  is  a  linear  set. 

Let  us  make  a  comment  on  the  case  t  =  1  that  is  not  covered  by  Proposition  3.1.  It 
is  clear  that  Te(x)  =  Gm(x )  with  some  m  and,  therefore,  G.m(x )  — *  x  as  m  — *  oo  implies 
Te(x)  — *  x  as  e  — *  0.  It  is  also  not  difficult  to  understand  that  in  general  Te(x)  — *  x  as 
e  — *  0  does  not  imply  Gm(x)  — *  x  as  m  — *  oo.  This  can  be  done,  for  instance,  considering 
the  trigonometric  system  in  the  space  Lp,  p  ^  2,  and  using  the  Rudin-Shapiro  polynomials 
(see  [T5]).  However,  if  for  the  trigonometric  system  we  put  the  Fourier  coefficients  with 
equal  absolute  values  in  a  natural  order  (say,  lexicographic),  then  in  the  case  1  <  p  <  oo 
by  Riesz  theorem  we  obtain  convergence  of  Gm(f)  from  convergence  of  Te(f).  Results  from 
the  paper  [KS]  show  that  the  situation  is  different  for  p  =  1.  In  this  case  the  natural  order 
does  not  help  to  derive  convergence  of  Gm(f)  from  convergence  of  Te(f). 

Let  us  give  an  application  of  the  results  of  this  section  for  summation  of  number  series. 
A  series  an ,  an  G  C,  is  said  to  A-converge  to  a  number  s  G  C  if  the  following  conditions 
hold: 


n:\an\>£ 


lim  e\{n  :  \an\  >  e}|  =  0. 
£^0+ 


(3.13) 
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We  shall  write  it  as 


(J)E 


an  =  s. 


The  notion  of  T-convergent  series  has  been  studied  in  [U 2] ;  see  also  [U3].  It  is  similar  to 
the  well-known  notion  of  the  T-integral  (see,  e.g.,  [Ul]).  We  show  that  T-convergence  can 
be  treated  as  weak  thresholding  convergence  of  number  series.  Recall  that  Co  is  the  space 
of  sequences  convergent  to  zero.  Namely, 

Cq  =  [x  =  (a:0,  x1, . . . )  :  xn  G  C,  lim  xn  =  ol  , 

l  n— >-00  J 

with  the  norm  of  x  G  Cq  defined  as  ||a:||  =  max„  \xn\.  It  is  known  that 


\x  <  00 


Co  =  h  =  \  ,  x1, . . . )  :  xn  G  C,  ||a:||  = 

l  71=0 

Consider  the  system  {en}ne^  C  Co  defined  as  e°  =  e™  =  1,  eJn  =  0  for  j  /  0,  n.  It  is 
clear  that  {en}  is  a  minimal  system.  It  is  also  easy  to  see  that  { en }  is  complete  in  cq.  For 
instance,  we  have  for  the  coordinate  vectors  un  ( u ™  =  1,  uJn  =  0,  j  ^  n),  n  =  0, 1, ... : 


^  m 

||«o - y^ejco  <  !/to; 


n=  1 


un  —  en  rto,  n  —  1,2,.... 

The  elements  e*  of  the  conjugate  system  are  e*  =  un,  n  =  1,2,....  Thus,  the  formal 
expansion  (3.3)  takes  the  form 


x 


E 

n=  1 


x  er 


Clearly,  this  expansion  converges  to  x  for  x  G  Co  satisfying  the  following  condition 

OO 

x0  =  J2xU- 


n=  1 


Theorem  3.3.  Define  the  system  {en}neN  C  Co  as  e°  =  e™  =  1,  eJn  =  0  /or  j  7^  0,  n. 
Tet  XX<en  an  be  a  number  series,  lim^-^  an  =  0,  s  G  C,  t  G  (0,1).  Then  the  following 
conditions  are  equivalent: 

1 )  the  series  XX  an  A-converges  to  s; 

2)  lime^0supDcDtj£  I T£,d  ~  s\  =  0,  where 

Dt,e  =  {j  ■  te  <  | afi  <  er},  T£yD  =  +  aA 

the  element  x  G  Co  defined  as  x  =  (s,  a-y,  02, . . . )  belongs  to  WT{en}. 

The  following  corollary  of  Theorems  3.2  and  3.3  has  been  proved  in  [U2]. 
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Corollary  3.1.  The  set  of  A-convergent  series  is  linear.  Moreover, 

(A)  ^^(an  +  bn )  =  (A)  an  +  (A)  b.n. 

n  n  n 

We  have  already  made  some  remarks  justifying  consideration  of  minimal  systems  instead 
of  bases  in  the  study  of  greedy  type  algorithms.  We  will  make  a  remark  (see  [KT2])  showing 
that  the  step  from  Banach  spaces  to  quasi-Banach  spaces  is  also  natural  in  studying  greedy 
type  algorithms. 

Remark  3.1.  One  can  check  (see  the  proof  of  Theorem  3.1  in  [KT2])  that  for  any  t  G  (0, 1) 
the  quasi-norm  |||  •  |||t  in  the  space  Y  =  WT{en}  G  Co  defined  as 

|||z|||t  =  sup  sup  \\T^D(x)\\ 

s  DCDti£(i) 


is  equivalent  to  the  quasi-norm 

|||x|||  =  max(|cc0|,  supe|{n  >  1  :  \xn\  >  e}|). 

£ 

Also,  a  quasi-norm  in  the  space  Y  can  be  treated  as  a  quasi-norm  in  the  space  of  A-convergent 
series. 

Theorem  3.4.  The  quasi-norm  |||  •  |||  in  the  space  Y  =  WT{en}  G  Co  is  not  equivalent  to 
any  norm. 

Proof.  It  is  sufficient  to  show  that  for  any  M  >  0  there  exist  a  positive  integer  m  and 
elements  x\, . . . ,  xm  from  Y  such  that 


(3.14)  |||a:j|||  <1  (j  =  1, . . .  ,m) 

and 


(3.15) 


1 

m 


j= i 


>  M. 


Take  an  even  to  G  N  and  set  xf  =  0  for  n  >  to,  xTf  =  (— 1  )n/k  for  1  <  n  <  m  where 
k  G  {1, . .  • ,  to}  is  defined  as  k  =  n  +  j(rnodm),  =  ^^=1  xj  -  ^  easy  see  a^ 
the  elements  Xj  =  (xj,Xj,...)  satisfy  (3.14).  Further,  for  the  element  x  =  ^  ]Cj=i  xj  = 
(x°,  x1, . . . )  we  have 

m 

\xn\  =  — V  l/k  (n  =  l,...,m). 

TO 

k=  1 

Therefore,  |||x|||  >  l/^5  an(l  (3.15)  holds  for  sufficiently  large  to.  The  proof  of  Theorem 

3.4  is  complete. 

The  reader  can  also  find  in  [KT2,  S.4]  applications  of  these  results  for  studying  A- 
convergence  of  trigonometric  series. 
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